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The  Scientist  as  Problem  Solver 

Hefbert  A  Simon 
Carnegie-Meiion  university 

The  thesis  o<  this  paper  can  oe  stated  succinctly  simply  by  replacing  the  "as" 
in  its  title  by  is  a'  The  Scientist  is  a  problem  solver,  if  the  thesis  'S  true  then 
we  can  dispense  with  a  theory  of  scientific  discovery  --  the  processes  of  discovery 
are  )ust  applications  of  fne  processes  of  problem  solving  However,  since  the  thesis 
IS  not  Obvious  to  everyone,  and  since  me  topic  of  scientific  discovery  has  interest  m 
its  own  right,  perhaps  it  is  worth  while  saying  a  little  more  about  it 

in  a  recent  book  (Langley  Simon  Bradshaw,  and  Zyikow.  1987).  my  co-authors 
and  I  have  said  a  great  deal  more  about  discovery,  and  Deepak  Kulkarni  and  I  have 
added  yet  another  chapter  m  a  paper  that  has  been  submitted  for  journal  publication 
(Kulkarni  and  Simon,  i986)  There  is  no  need  to  repeal  these  accounts  here  beyond 
the  briefest  summary  of  what  we  concluded  and  how  we  supported  our  conclusions 
We  concluded  that  the  thesis  is.  indeed,  valid  As  evidence,  we  adduced  careful 
reports  of  a  substantial  number  of  historical  scientific  discoveries,  together  with 
computer  simulations  that,  starting  with  essentially  the  same  initial  conditions  as  did 
the  human  discoverers,  made  the  same  discoveries  Thus  the  computer  programs 
contained  a  set  of  processes  that  were  sufficient  for  making  me  discoveries,  and 
thereby  provided  a  possible  explanation  for  me  success  of  me  human  scientists 

Our  explorations  of  the  histones  of  sciontn.c  discoveries  have  made  it  eminently 
clear  to  us  that  scientists  set  themselves  many  different  kinds  of  tasks  These 
include  tasks  of  formulating  significant  scientific  problems,  of  discovering  interesting 
poenomena.  oi  finding  the  laws  mat  are  hidden  m  data  twith  and  witnout  me  help  of 
thcuMco  for  guiding  me  searcn).  oI  .nve.u.ng  ne'--  representations  fm  ohenomena  and 
their  accompanying  theories  of  inferring  the  logical  consequences  of  theories  and 
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testing  them  of  designing  experiments,  of  finding  explanatory  mechanisms  to  account 
for  empirical  generalizations,  and  of  inventing  new  instruments  for  observation  and 
measurement  Undoubtedly  there  are  others 

Wnat  iS  common  to  all  of  these  tasks  is  that  they  appear  to  employ  the  same 
general  Kinds  of  problem  solving  processes  as  are  employed  by  chessplayers  m 
choosing  moves  by  subjects  m  the  laboratory  confronted  with  the  Tower  of  Hanoi  or 
the  Missionaries  and  Cannibals  problem,  by  physicians  making  diagnoses,  by 
computer  salesmen  configuring  systems  for  clients,  by  architects  designing  houses,  or 
by  organic  chemists  Synthesizing  new  molecules  Mostly,  they  engage  in  heuristic 
search  in  a  number  of  problem  spaces:  the  spaces  of  theories  and  experiments 
mentioned  by  Klahr  and  Dunbar  (this  volume),  but  also  spaces  of  problems,  of 
phenomena,  of  representations,  of  instruments,  and  others 

Moreover,  the  "insight"  that  is  supposed  to  be  required  for  such  work  as 
discovery  turns  out  to  be  synonymous  with  the  familiar  process  of  recognition;  and 
other  terms  commonly  used  in  the  discussion  of  creative  work  ••  such  terms  as 
"judgment."  "creativity."  or  even  "genius"  -  appear  either  to  be  wholly  dispensable 
or  to  be  definable,  as  insight  is.  in  terms  of  mundane  and  well  understood  concepts. 

Until  rather  recent  times,  much  of  the  published  work  about  scientific  discovery 
has  consisted  of  anecdotes,  frequently  autobiographical,  about  specific  discoveries  and 
their  finders  If  discovery  requires  creativity,  or  even  genius,  it  would  be  immodest 
for  anyone  to  claim  that  he  or  she  had  made  a  discovery,  and  futile  to  try  to 
describe  how  it  had  been  done.  But  if  d  scovery  is  plain,  garden-variety  problem 
solving,  then  there  is  no  immodesty,  and  perhaps  not  even  futility  m  adding  to  the 
anecdotal  evidence  i  shall  use  this  opportunity  to  think  aloud  aibeit  retrospectively 
about  some  of  my  own  scientific  work,  ano  to  see  whether  it.  too.  fits  the  problem¬ 
solving  mold 
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I  say  "albeit  retrospectively  "  but  backward  preoicnons  are  really  the  only  ones 
we  can  trust  when  we  are  dealing  with  a  theory  of  human  behavior  After  an  when 
we  make  forward  predictions,  our  scientists  may  nave  been  influenced  by  the  very 
theories  of  discovery  we  are  trying  to  test  The  theory  may  fit  tneir  behavior  cniv 
because  t-oy  read  about  3ACON  or  DALTON,  and  thmk  they  will  do  better 

scence  if  they  simulate  those  programs  We  will  avoid  i>iat  danger  of  spurious 
verification  by  predicting  events  from  a  time  when  the  theory  did  not  exist 

Formulating  Problems 

If  IS  usually  thought  that  a  prerequisite  to  answering  a  question  is  to  state  it 
Or,  to  change  the  metaphor,  for  something  to  be  found,  something  must  have  been 
lost.  But  Is  that  always  true’’  When  one  finds  a  vein  of  gold,  was  it  Nature  who 
lost  if’  If  we  can  find  gold  we  haven  t  lost,  perhaps  we  can  answer  questions  wo 
haven't  asked 

Let's  try  again.  We  may  find  gold  (even  gold  we  haven't  lost)  by  searching  for 
it.  But  that  means  that  the  question  has  already  been  asked:  "Where  can  we  find 
some  gold’"  But  what  about  the  gold  we  find  when  we  are  not  looking  for  gold 
when  we  are  engaged  in  some  quite  different  activity  (gathering  wildflowers  on  the 
mountain,  say)’  At  the  very  least,  we  must  nonce  the  gold:  it  must  attract  our 

attention,  distracting  us  from  the  flowers  Do  we  account  for  this  by  postulating  a 
need  for  gold’  Or  will  an  attention-attracting  propensity  of  shmy  yellow  obiects  do 
the  )0b?  And  how  is  the  attraction  of  these  yellow  obiects  enabled  by  cur 

dis’ractabili’y  from  the  ftower-gathenng  task’ 

Now  let's  return  from  gold-seeking  to  problem-seeking  if  we  take  our 

metaphor  seriously,  it  suggests  that  one  way  to  find  a  problem,  and  perhaps  even  its 

solution  IS  to  try  to  solve  some  other  problem  That  doesn't  tell  us  where  the  othe' 
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problem  came  from,  but  one  problem  at  a  time'  lA/e  are  dealing  with  the 
phenomenon  of  surprise  Searching  for  wiidfiowers  we  are  surprised  to  see 

something  shmmg  and  golden  in  the  rocks  To  be  surprised  we  must  attend  to  me 
surprising  phenomenon  Hence  the  dictum  of  Pasteur  "Accidents  haopen  to  tne 
prepared  mmd."  And  now  we  have  a  new  problem  How  does  a  mmo  become 

prepared'!’  Perhaps  it  is  time  for  the  anecdote 

My  first  piece  of  scientific  work  begun  as  a  paper  in  an  "independent 
projects"  course  at  the  University  of  Chicago  m  the  Winter  and  Spring  of  1935.  was 
^  study  the  administration  of  public  recreation  in  the  City  of  Milwaukee  (Simon 
1935).  Never  mmd  why  that  was  a  problem  it  was  relevant  to  a  research  project  of 
my  professor,  Jerry  Kerwin.  on  the  relations  of  school  boards  with  city  governments 
A  standard  topic  in  studies  of  organizations  is  the  budget  process,  which  in  this  case 
involved  the  division  of  funds  between  playground  maintenance,  administered  by  one 
organizational  unit,  and  playground  activity  leadership,  administered  by  another.  How 
was  this  division  (which  was  a  frequent  subiect  of  dispute)  arrived  at? 

My  previous  study  of  economics  provided  a  ready  hypothesis;  divide  the  funds 
so  that  the  next  dollar  spent  for  physical  maintenance  would  produce  the  same 
return  as  the  next  dollar  spent  for  leaders  salaries  i  saw  no  evidence  that  anyone 
was  viewing  the  decision  in  this  way  Was  i  surprised’’  Perhaps,  initially,  but  on 
reflection,  i  didn't  see  how  it  could  be  done  How  were  the  values  of  better  activity 
leadership  to  be  weighed  agamst  the  values  of  more  attractive  and  better-mamiamed 
neighborhood  playgrounds’’ 

Now  I  had  a  new  research  problem  how  do  human  bemgs  reason  when  the 
conditior^s  for  rationality  postulated  by  the  model  of  neoclassical  economics  are  not 
met  --  for  example,  when  no  one  can  define  the  appropriate  utility  function  or 
Suggest  how  the  contribution  of  expenditures  to  utility  is  to  be  measured’ 
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Investigating  funher  the  panicular  situation  before  me  i  thought  i  could  see  a  rathe, 
simple  pattern  of  the  mental  processes  Those  who  were  organirationaiiy  responsible 
for  playground  supervision  wanted  more  money  spent  for  leadership  those  who  we^e 
responsible  for  the  physical  condition  ot  the  playgrounds  wanted  more  spent  for 
maintenance  Generalizing,  people  m  organizations  bring  decision  problems  within 
reasonable  bounds  by  identifying  with  the  partial  (and  more  nearly  operational)  goals 
that  are  the  particular  responsibility  of  their  own  organizational  units  (Simon  1947 
ch  10), 

Of  course  this  is  only  a  partial  answer  It  defined  and  labeled  the 
phenomenon  of  organizational  identification,  a  concept  that  has  proved  valuable  m 
administrative  theory,  but  it  did  not  explain  how  higher  levels  of  the  organization 
adjudicated  between  the  claims  arising  from  competing  identifications  at  the  lower 
levels.  That  subject  has  subsequently  been  addressed  by  other  researchers,  among 
them  John  P.  Crecine.  who  wrote  his  dissertation  on  this  topic  some  thirty  years  after 
the  events  i  am  describing  (Creeme  1969) 

The  broader  question  -  how  do  people  make  decisions  when  the  conditions  for 
the  economists  global  rationality  are  not  met  (or  even  when  they  are)”^  -•  remains  an 
active  frontier  of  research  today,  although  large  oieces  of  an  answer  have  been 
provided  through  research  by  cognitive  scientists  on  problem  solving  Here  the 
centra!  concept  is  what  economists  call  "bounded  rationality,"  and  what  cognitive 
scientists  would  more  likely  label  "computational  constraints  on  human  thinking  "  A 
large  part  of  the  answer  is  that  when  people  con  t  know  how  to  optimize,  they  may 
very  well  be  able  to  satisfice  ■■  to  fmo  gooo-enough  solutions  And  good-enougn 
solutions  can  often  be  found  by  'cunstic  search  (Simon.  1955  i982) 

Now  w'hat  does  this  anecdote  say  about  finding  problems  as  an  essential 
component  m  the  process  of  scientific  discovery^  One  thing  it  says  is  that  a 
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problem  I  found  m  1935  has  lasted  me  for  fifty  two  years  i  have  never  had  to  find 
another.  More  accurately,  this  very  broad  problem  of  accounting  for  human 
rationality  has  served  as  a  powerful  generator  for  an  endless  senes  of  suborobiems 
(e  g  .  how  do  people  sowe  the  Tower  of  Hanoi  problem  now  do  they  choose  chess 
moves,  how  do  they  make  scientific  discoveries^)  (Newell  and  Simon,  1972  Simon 
1979.  sections  4  7,  Langley  et  al  1987). 

Another  lesson  to  be  drawn  from  tne  anecdote  is  that  scientific  discovery  is 
incremental  An  explanation  for  a  particular  act  of  discovery  must  take  everything 

that  has  gone  before  as  initial  conditions  What  we  seek  to  explain  Is  how  these 
initial  conditions  led  to  the  next  step  --  m  this  case  how  my  knowledge  of  elementary 
price  theory,  and  Jerry  Kerwin  s  desire  to  know  how  the  school  board  and  the  public 

works  department  cooperated  to  provide  public  recreation  services  m  Milwaukee,  led 

me  to  observe  a  phenomenon  that  initially  surprised  me:  and  how  that  surprise  led  to 
new  observations  that  could  be  explained  by  the  concepts  of  identification  and 

bounded  rationality  Steps  taken  twenty  years  later  led  from  bounded  rationality  to 
satisficing,  and  from  satisficing  to  heuristic  search 

Third,  the  anecdote  adds  another  to  the  long  list  of  examples  where  surprise 
was  a  key  element  m  discovery.  But  what  was  "prepared'  about  this  particular 
mind'^  My  training  in  economics,  and  the  evocation  of  that  training  m  the  context  of 
a  budget  situation,  disclosed  a  contradiction  between  what  theory  taught  me  ought  to 
be  happening  and  what  my  eyes  and  ears  showed  me  to  be  actually  happening 
Without  the  training  in  economics  the  observed  behavior  would  have  appeared  entirely 
"natural  "  Without  the  observations.  I  could  have  continued  m  the  happy  illusion  that 
the  neoclassical  theory  of  utility  maximization  explains  human  behavior  in  the  domain 
of  budgeting 

Nothing  mystical  Nothing  magical.  Can  we  simulate  it’  The  heuristics  indeed 
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resemble  quite  doseiy  those  o<  KEKADA.  the  program  that  Deepak  Kuikarm  and  i 

have  used  to  simulate  the  research  strategy  ot  Hans  Kreps  who  found  the  chemical 
path  for  the  'h  v/vo  synthesis  of  urea,  a  prog'^am  mat  has  now  been  generalized  to 
other  discoveries  (Kuikarm  and  Simon  1986)  The  program  experiences  surprise 

when  Its  expectations  are  not  met  ana  reacts  to  its  surprise  by  seeking  explanations 

for  the  surprising  phenomena  But  now  I  am  waving  my  hands  (Or  am  I  hand 

Simulating'’)  We  have  not  yet  investigated  what  heuristics  would  have  to  be  added 
to  KEKADA  in  order  to  Simulate  the  discovery  of  bounded  rationality.  But  i  think  i 

might  have  saved  myself  a  lot  of  work  m  i935  if  I  had  had  KEKADA  to  advise  me 

Laws  from  Data 

In  our  book  Scientific  Discovery  (Langley  et  al  1987).  my  colleagues  and  l 

gave  primary  attention  to  the  problems  of  inducing  generalizations,  quantitative  and 
qualitative  from  empirical  data  Our  programs  BACON  and  DALTON,  were  systems 
of  heuristics  for  inducing  Quantitative  laws,  and  our  programs  STAHL  and  GLAUBER 
systems  for  inducing  qualitative  laws 

Data  are  not  the  only  possible  initial  conditions  for  the  induction  of  new  laws, 
theories  can  also  be  used,  m  conjunction  with  data  or  independently  in  our  BACON 
simulations  we  showed  that  by  incorporating  m  BACON  heunsucs  that  search  for 
symmetries  and  conservation  laws,  we  could  substantially  improve  the  efficiency  with 
which  It  found  laws  m  empirical  data  m  the  limit,  it  may  be  possible  to  find  a 

descriptive  law  directly,  by  deriving  it  from  a  more  fundamental  explanato'y  law  For 
example  Newton  showed  that  Kepler  s  Third  Law  of  Planetary  Motion  (the  cenod  cf 
revolution  of  a  planet  vanes  as  the  3'2  power  of  ns  distance  from  the  Suni  could  be 
derived  mathematically  from  the  inverse  square  law  of  gravitational  attraction  (But 
note  that  Newton  was  working  backward  from  the  law  that  Kepler  had  already 
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discovered  by  data-driven  search  ) 

Before  one  can  find  mathematical  functions  that  fit  empirical  data,  one  must 
have  apDropriate  data  that  look  as  though  a  smooth  matnem.atica'  function  couid 
generate  tn.em  it  s  the  recipe  for  rabbit  stew  ail  over  again  first  catch  me  rappi’ 
Examples  of  suen  data  nave  been  much  easier  to  come  by  m  the  physical  sciences 
than  in  the  biological  or  social  sciences  When  we  find  social  science  data  of  this 
kind,  we  should  prize  them 

On  only  one  occasion  m  my  life  have  I  run  on  to  suen  data,  and  i  cannot 
recall  exactly  when  i  first  encountered  them  -  possibly  as  early  as  about  1936  in 
Lotka's  Elements  of  nhysical  Biology  (1924),  a  fascinating  book  that  the  economist 
Henry  Schultz  always  called  to  the  attention  of  his  students  Lotka  reports  data, 
compiled  by  one  Dr  J  C.  Willis,  showing  that  when  the  number  of  species 

belonging  to  each  genus  m  some  order  of  plants  or  animals  (beetles,  say)  are 
counted,  and  the  genera  are  then  arranged  in  order,  according  to  the  number  of 
their  species,  the  genus  with  the  nth  largest  number  of  species  will  have  about  l/n 
as  many  species  as  the  genus  with  the  largest  number 

Similarly,  when  the  frequencies  with  which  diffe  ent  words  appear  m  a  book  are 
counted,  and  words  are  then  arranged  m  order  of  their  freauency.  the  nth  most 
frequent  word  will  occui  about  l/n  times  as  frequently  as  the  most  frequent  woid 
Moreover  about  half  of  all  the  words  that  occur  m  a  book  will  occur  exactly  once 
about  one  sixth  exactly  twice  one  twelfth  three  times  and  so  on  These  relations 
hold  for  books  m  any  alphabetic  language  and  t.ie  deoa'tures  from  regularity  are 
small 

Other  data  show  a  similar  regularity  m  me  populations  cf  cties  m  ihe  United 
States  the  nth  largest  ciiy  is  about  i/n  times  as  large  as  New  York  These 
regularities  are  easily  seen  if  the  data  are  plotted  on  log-log  paper,  whereupon  they 
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fall  on  a  straigni  ime  with  a  slope  of  minus  one 

What  does  one  do  with  regularities  like  this  --  regularities  that  at  first  blush 

can  only  be  described  as  astonishing'’  What  one  does  (or  should  do)  is  to  behave 
m  a  BACON-iike  fashion  until  one  finds  a  formula  that  fits  the  data  Then  like 

DALTON  one  sncuid  see  if  one  can  postulate  a  mechanism  whose  operation  would 
produce  the  regularity  described  by  the  formula 

I  wish  I  could  say  that  this  was  my  immediate  response  to  the  data  Memory 

fails  me  i  recall  my  fascination  with  them  but  not  whether  I  pondered  over  them, 

and  if  I  did  for  how  long  I  do  recall  that,  when  I  returned  to  Chicago  after  1942, 

I  thought  about  them  again  --  i  nave  a  clear  picture  of  sitting  in  the  Biology  Library 

in  the  Univers  ty  of  Chicago  reading  a  paper  referenced  in  Lotka  s  book.  l  also 
recall  mentioning  my  interest  in  the  matter  to  Allen  Newell  while  visiting  him  and  'Weet 
in  their  Santa  Momca  apartment  between  1952  and  1954.  But  I  was  doing  many 
other  things  during  these  years  The  startling  data  on  word  frequencies  and  city 
sizes  were  not  a  constant  preoccupation,  but  were  more  i;ke  a  recurring  itch  that 
needed  to  be  scratched  occasionally 

Sometime  during  1954  i  found  the  answer  My  recollections  of  just  how  i 

found  It  are  sketchy,  with  no  scraps  of  paper  to  bolster  my  memory,  but  a  few 

aspects  of  the  discovery  are  recoverable  now  First  i  looked  for  a  function  to  fit  the 
data  I  was  especially  impressed  by  the  regularity  of  the  word-frequency  data  at  the 
low  end  of  the  frequency  range  '’"he  simple  fractions  seemed  to  point  to  a  formula 
involving  ratios  of  integers  m  fact  the  simple  formula  f(i)  =  i/Ii(i-l)].  gives  the 
required  numbers.  i/2  i/6  i-'i2,  ano  so  on  For  large  i  we  have  approximately,  f(i) 
=  i/i^  The  rank  which  is  simply  the  integral  of  the  frequency  will  then  give  F(i)  =z 
i'i  so  that  on  a  logarithmic  scaie  the  relation  between  rank  ano  frequency  will  be 
linear  with  a  slope  of  minus  one 
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Finding  an  eauation  that  fits  these  magic  numbers  sets  the  stage  for  a  new 

proDiem:  finding  an  explanation  for  the  equation  a  plausible  mechanism  that  wiii 

provide  a  rationale  for  the  phenomena  My  recollections  of  now  i  did  this  are  even 

sketchier  than  my  recollections  about  the  previous  stage  fbe  ratios  of  integers  were 

again  the  key  Where  can  you  get  ratios  of  integers’  Ratios  of  factorials  are  one 

possible  source  if6  can  be  written  as  me  product  of  i/2  and  i/3.  and  in2  as  the 

product  of  1/2.  1/3.  and  2/4  in  general,  the  formula  (1-1)1/11  produces  the  required 
numbers  The  next  step  is  likely  to  occur  only  to  someone  who  has  a  little 

mathematical  knowledge,  and  who  sees  in  these  ratios  of  factorials  something  like  the 
Beta  function,  or  at  least  sees  the  kinds  of  expressions  one  is  accustomed  to 
encounter  in  problems  on  combinations  and  probabilities  (In  fact.  I  discovered  that 

the  Beta  function  was  wnat  l  wanted  by  searching  through  my  copy  of  Peirce  s  A 
Short  Table  of  integrals,  where  i  vaguely  remembered  having  seen  some  ratios  of 

factorials  ) 

Are  there  any  other  reasons  for  thinking  that  the  situation  may  call  for  a 

probability  model’  indeed  there  are  What  do  word  frequency  distributions  and  city 
size  distributions  (as  well  as  a  number  of  quite  different  phenomena  where  this  same 
law  applies)  have  m  common’  Nothing  very  obvious,  unless  they  can  be  viewed  as 
instantiations  of  the  same  urn  scheme  So  let  us  see  whether  we  can  interpret  the 
formula  as  representing  the  steady  state  of  some  stochastic  process 

Here  I  recall  being  aided  by  a  metaphor  if  we  thmk  of  a  book  as  being 

created  word  by  word,  and  if  a  word  is  added  that  has  already  occured  k  times,  the 
number  of  words  cccunng  k -*■  1  times  each  will  be  increased  by  one  and  the  number 
of  words  occuring  k  times  each  decreased  by  one  For  a  steady-state  equilibrium 
the  rate  at  which  words  are  created  that  had  previously  occured  k  times  must  be 

equal  to  the  rate  at  which  words  are  created  that  had  previously  occured  k-i  times 
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In  this  way  the  "k"  bm  will  be  replenished  as  rapidly  as  it  is  depleted  At  some 
point  I  began  to  visualize  this  as  a  cascade  with  successive  pools  of  water  each 
maintained  at  a  constant  level  by  flow  m  from  the  pool  above,  and  flow  out  to  the 
pool  next  below  Working  back  from  our  answer  --  the  distribution  that  we  know 
describes  the  phenomena  -•  it  is  not  too  hard  to  show  that  the  equilibrium  condition 
requires  that  the  probability  of  creating  a  word  that  has  already  occured  k  times 
must  be  proportional  to  k 

We  are  ready  for  the  final  step  to  interpret  the  probability  assumption.  In  the 
case  of  word  distributions,  it  can  be  interpreted  to  mean  that  the  chance  of  a  word 
being  chosen  as  the  next  word  in  a  text  is  proportional,  because  of  association,  to 
the  frequency  with  which  it  has  been  used  already,  and  also  proportional,  because  of 
long-term  associations  stored  in  memory,  to  the  ireauency  with  which  It  is  used  in 
the  language.  In  the  case  of  city  sizes,  it  can  be  interpreted  to  mean  that  birth  and 
death  rates  are  approximately  independent  of  city  size,  while  the  probability  that  a 
city  will  be  the  target  for  any  given  migration  is  also  proportional  to  its  size  (Simon. 
1955) 

I  don't  propose  to  defend  these  interpretations  here  My  purpose  is  to 
understand  tne  process  that  reached  them  if  my  accourl  through  the  filter  of  thirty 
to  fifty  years  of  forgetting  has  any  relation  to  reality,  then  we  see  a  process  for 
arriving  at  the  initial  formula  that  looks  very  BACON-hke  followed  by  working-backward 
search  processes  that  are  guided  by  the  evocation  of  prestored  mathematical  and 
real-world  knowledge  -  BACON  as  the  front  end  to  an  expert  system 

Again,  my  hands  are  waving  wildly  You  will  not  have  failed  to  notice  that  i 
have  not  accounted  at  all  for  the  cascade  metaphor,  yet  at  some  time  n  was  evoked 
and  helped  me  to  formulate  the  steady-state  relations  So  there  is  still  work  to  be 
done  on  the  theory  of  discovery  still  theses  to  be  written  and  papers  published  But 
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I  see  in  this  little  history,  or  imagined  history,  no  magic  and  no  mystery  Each  step 
appears  to  proceed,  if  not  inexorably  at  least  plausibly  from  the  preceding  one 

If  the  data  cried  out  so  loudly  for  explanation,  and  if  the  discovery  process 

proceeded  in  such  a  plausible  succession  of  steps  why  did  not  others  discover  this 

law  and  Its  stochastic  explanation’’  indeed  they  did  The  first  was  G.  Udny  Yule. 

the  English  statistician,  who  m  1924  published  "A  mathematical  theory  of  evolution 

based  on  the  conciusions  of  Dr  j.  C  Willis.  F  R  S."  Yule  constructed  a  model  very 

similar  to  the  one  I  have  just  described  to  explain  Willis’  data,  mentioned  earlier,  on 

the  distribution  of  species  among  genera.  (I  could  have  been  led  to  this  paper  by  a 

footnote  In  Lotka.  but  I  wasn't.)  A  second  was  the  English  economist. 

D.  G.  Champernowne.  who  published  "A  model  of  income  distribution"  in  1953. 

describing  a  quite  similar  process.  A  third  was  B.  Mandelbrot,  who,  in  1953. 

published  "An  informational  theory  of  the  statistical  structure  of  language."  There 

were  some  differences  between  Mandelbrot's  model  and  mine,  which  later  occasioned 

heated  dispute  between  us,  but  the  basic  ideas  were  closely  related. 

A 

I  learned  about  all  of  these  partial  anticipations  when  l  searched  the  literature 
and  inquired  among  my  friends  prior  to  publishing,  m  1955.  my  own  paper  on  the 
topic  (Simon.  1955). 

That  still  isn  I  quite  the  end  of  the  story,  for  again,  the  solution  of  one 
scientific  problem  created  a  host  of  new  problems  in  the  book  by  Yuji  Ijin  and 
myself.  Skew  DistnPunons  and  the  Sizes  of  Busir^ess  Firms  (1977),  you  can  fmd  a 
series  of  essays  applying  the  same  stochastic  mechanism,  and  generalized  versions  of 
it.  to  the  task  of  unoerstanding  the  size  distributions  of  business  firms  and  the 
economic  implications  of  these  distributions 
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Representations 

Mention  of  the  cascade  metaohor  that  i  used  to  fmd  the  stochastic  process 

underlying  word-freouency  and  city-size  distributions  raises  the  question  of 
representations  What  kinds  of  representations  are  used  by  scientists  m  thinking 
about  their  research  problems,  and  where  do  these  representations  come  from'^  One 
hallowed  form  of  the  question  is  whether  scientists  (and  others)  think  in  words,  or 
whether  thoughts  take  some  quite  different  shape  --  whether  they  employ  “mental 
pictures,"  say 

Words  and  Pictures 

The  French  mathematician.  Jacques  Hadamard,  in  his  delightful  essay  on  The 

Psychology  of  Invention  m  the  Mathematical  field  (1945).  comes  down  heavily  on  the 

side  of  images  and  against  words  Among  the  many  distinguished  mathematicians 

and  scientists  testifying  for  him  is  Albert  Einstein,  who  in  a  letter  to  Hadamard  stated 
that  "the  words  or  the  language,  as  they  are  written  or  spoken,  do  not  seem  to  play 

any  role  in  my  mechanism  of  thought.  The  psychical  entities  which  seem  to  serve 

as  elements  in  thought  are  certain  signs  and  more  or  less  clear  images  which  can 
be  'voluntarily  reproduced  and  combined." 

What  is  good  enough  for  Hadamard  and  Emstem  is  good  enough  for  me  i 
too.  have  difficulty  m  finding  any  presence  of  words  when  i  am  thinking  about 

difficult  matters,  especially  mathematical  ones.  Even  as  i  sit  here  at  the  keyboard 
composing  this  essay.  I  cannot  really  detect  the  words  m  my  thoughts  (or  much  cf 
anything  else,  for  that  matter;  until  they  come  out  the  ends  of  my  fingers  But 
perhaps  i  am  not  thinking,  but  just  recording  previously  composed  ideas  that  reside 
somewhere  m  my  subconscious  mmd 

Even  if  we  do  think  in  images  rather  than  words,  neither  Hadamard  nor 
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Einstein  nor  i  had  much  success  m  describing  just  what  these  images  were  or  how 
they  were  represented  in  a  biological  structure  like  the  brain  However  i  believe  that 
Jill  Larkin  and  i  have  recently  made  substantial  progress  m  explaining  these  matters 
(in  wny  a  Diagram  is  (Sometimes)  Worm  Ten  Thousand  Woras)  (LarKin  and  Simon. 
1987)  In  order  to  deal  with  the  difficulties  one  by  one  we  fudged  a  bit  alleging 
that  we  were  talking  about  diagrams  on  paper  rather  than  mental  pictures,  but  most 
of  bur  argument  carries  over  m  a  straightforward  way  The  basic  ideas,  which  I  will 
not  elaborate  upon  here  are  (a)  that  m  the  course  of  transforming  verbal  propositions 
into  images,  many  things  are  made  explicit  that  were  previously  implicit  and  hidden, 
and  (b)  that  (learned)  inference  operators  facilitate  making  additional  inferences  from 
the  images  In  computationally  efficient  ways 

We  also  show,  as  a  byproduct  of  our  analysis,  that  diagrams  are  representable 
as  list  structures  (alias  "schemas."  "scripts."  "frames."  "labeled  directed  graphs."  et 
cetera),  hence  are  programmable  in  standard  list-processing  languages,  hence  are 
readily  seen  to  be  representable  in  systems  of  neuron-like  structures.  Since  the 
surface  structures  and  the  semantics  of  natural  languages  can  also  be  represented 
as  list  structures,  we  can  conclude  that  propositions  and  pictures  (or  at  least 
diagrams)  can  use  common  machinery  in  the  brain  -  that  both  are  best  viewed  as 
specializations  of  a  common  list-structure  mode  of  representation  (I  do  not  wish  to 
deny  that  we  may  also,  as  Kossiyn  (1980.  and  also  this  volume)  has  argued,  possess 
a  specialized  raster-like  organ  for  more  literal  representation  of  visual  images  --  a 
mind  s  eye.  But  I  would  prefer  to  put  that  Question  aside  here  ) 

Now  )ust  as  there  has  long  been  a  debate  as  to  whether  we  use  words  or 
images  m  our  thoughts,  so  there  has  been  a  debate  (perhaps  the  same  debate)  as 
to  whether  our  internal  represe.''tations  of  problems  look  like  collections  of 
propositions  or  like  models  of  the  problem  situations  Each  of  these  views  has  been 
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held  by  an  important  segment  ot  the  cognitive  science  community  and  the  two 
segments  do  not  often  communicate  with  each  other  except  sometimes  to  quarrel 

One  segment,  under  the  banner  of  "let  language  lead  tne  way  "  takes  verbal 
reasoning  as  i*s  me'aphor  for  the  problem-soivmg  process  and  thinks  of  reasoning  as 
some  kind  ct  PROLOG-like  theorem-provmg  procedure  The  book  by  Miller  and 
Johnson-Laird  on  Perception  ana  Language  (1976)  is  an  excellent  representative  of 
this  point  of  view,  although  Johnson-Laird.  m  his  more  recent  book.  Mental  Models 
(1987).  takes  a  long  step  of  apostacy  toward  the  alternative  viewpoint.  That  he  does 
so  without  any  apparent  awareness  that  he  is  moving  onto  well-explored  ground 
exemplifies  the  mutual  insulation  of  the  two  segments 

The  second  segment  of  the  cognitive  science  community  uses  heuristic  search 
through  a  problem  space  (a  mental  model  of  the  task  domain)  as  its  metaphor  for 
problem  solving  Human  Problem  Solving  (1972)  adheres  strictly  to  this  viewpoint  it 
has  been  claimed,  by  Pylyshyn  (1973)  among  others,  that  the  two  viewpoints  cannot 
be  distinguished  operationally,  but  this  claim  rests  on  a  confusion  between  the 
informational  equivalence  and  the  computational  equivalence  of  representations  Ev»n 
if  two  representations  contain  exactly  the  same  information,  it  may  be  far  cheaper 
computationally,  to  make  some  of  this  information  explicit  using  one  rerpresentation 
than  using  the  other.  The  incorrectness  of  the  claim  of  computational  equivalence  is 
demonstrated  by  the  examples  given  m  the  Larkm-Simon  paper  mentioned  above 

Reoresenting  a  Dynamic  System 

I  am  afraid  that  l  have  been  diverted  from  my  mam  topic,  which  is  providing 
anecdotal  evidence  about  the  problem  solving  processes  used  m  scientific  discovcv 
Let  me  return  with  an  example  that  I  will  present  rather  sketchily,  to  avoid  technical 
detail  Economists  frequently  use  what  they  call  "partial  equilibrium  analysis  m 
which  they  avoid  talking  about  ever,  -mg  at  once  by  making  a  host  of  cetens  panbus 
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assumptions  They  examine  the  impact  of  a  disturbance  upon  a  small  segment  of 
the  economy  while  assuming  no  interaction  with  the  rest  of  the  economy 

(f  challenged  on  the  legitimacy  of  what  they  are  doing,  economists  using  partial 
equilibrium  methods  may  defend  themselves  by  saying  that,  of  course,  interactions  are 
not  completely  absent  but  they  are  small,  hence  unimportant  That  is  an  argument 
we  know  not  only  m  economics,  but  throughout  all  of  science  But  is  it  a 

satisfactory  argument'’  Small  effects,  persisting  over  a  long  period  of  time,  may 
integrate  into  large  effects 

Thoughts  of  these  kinds  (represented  as  words  or  as  images'’)  went  through  my 
mind  while  I  read,  in  the  early  I950s.  a  paper  by  Richard  Goodwin.  Dynamic 

Coupling  with  Especial  Reference  to  Markets  Having  Production  Lags,  published  in 
Econametrica  in  1947  Again  l  cannot  claim  any  clear  recollection  of  the  precise 
steps  I  took  to  formulate  and  solve  the  problem  that  his  paper  evoked  i  did 
conceive  of  it  as  a  matter  of  analysing  the  behavior  of  a  large  dynamic  system 
divided  into  sectors,  with  strong  interactions  among  the  components  in  each  sector 
and  weak  interactions  among  sectors.  I  remember  also  that  I  worked  very  hard  for 
several  months  to  get  answers,  and  that  l  worked,  without  paper  and  pencil,  while 
taking  long  walks 

My  representation  at  least  much  of  the  time,  was  an  image  of  the  matrix  of 
coefficients  of  such  a  dynamic  system  -  hardly  surprising,  since  this  is  the  way 
dynamic  systems  are  normally  represented  m  mathematics  books  At  some  point  i 
saw  that  the  rows  and  columns  of  the  matrix  could  be  permuted  so  that  the  new 

matrix  would  consist  of  a  number  of  diagonal  blocks  with  large  coefficients  m  them 

and  only  small  coefficients  in  the  matrix  outside  the  diagonal  blocks  The  matrix  was 
"nearly  block  diagonal  "  The  image  was  vague,  m  that  tne  number  of  blocks  arid 
their  sizes  were  not  seen  m  detail  if  forced  to  give  numbers.  I  might  say  that  there 


16 


The  Scieniisi  as  Problem  Solver 


1  July  1987 


could  have  been  three  blocks,  each  three  rows  by  three  columns  m  size  --  but  the 
answer  is  surely  a  fabrication. 

At  som'e  later  point  m  nme.  I  acquired  a  metaphor  l  visualized  a  building 
divided  into  rooms,  each  of  which  was  divided,  in  turn,  into  cubicles  You  can  see  a 
diagrammatic  interpretation  of  my  metaphor  on  page  2l2  of  The  Sciences  of  the 
Artificial  (1981)  Second  Edition  We  start  out  with  an  extreme  disequilibrium  of 
temperature  each  cubic  foot  of  each  cubicle  being  at  a  different  temperature  from 
its  neighbors 

Several  things  now  seemed  obvious.  Throughout  each  cubicle,  a  constant 
temperature  would  be  established  very  rapidly  by  exchange  of  heat  between  adjoining 
volumes  At  some  later  time,  each  room  would  attain  a  constant  temperature  by 
heat  diffusion  through  the  walls  of  the  cubicles  a  still  later  tire,  the  entire 

building  would  reacn  a  constant  temperature  by  exchange  of  heat  between  the 

thicker  walls  of  the  rooms 

Moreover,  because  of  the  differences  in  the  durations  involved,  each  of  these 
processes  of  equilibration  -•  within  cubicles,  among  cubicles,  and  among  rooms  ••  can 
be  studied  independently  of  the  others  in  studying  the  equilibration  of  each  cubicle, 
we  can  ignore  the  other  cubicles,  in  studying  the  equilibration  of  rooms,  we  can 
represent  each  cubicle  by  its  average  temperature,  and  ignore  the  other  rooms  in 
studying  the  equilibration  of  the  building,  we  can  represent  each  room  by  its  average 
temperature  As  a  result,  the  mathematics  of  the  problem  can  be  simplified 

drastically 

There  still  were  some  difficult  mathematical  steps  from  this  picture  of  the 
situation  to  rigorous  proofs  of  the  (approximate)  validity  of  the  simplification,  but  the 
result  to  be  attained  was  clear  The  reasoning  i  have  described  was  earned  out 
mainly  m  the  summer  of  1956.  and  incorporated,  together  with  the  mathematics,  in  a 
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paper  written  with  Albert  Ando  later  that  year,  and  published  m  1961  (Simon  and 
Ando.  1961) 

I  can  throw  no  further  light  on  the  source  of  the  heat-exchange  metaphor  or 
on  now  if  at  all.  i  drew  inferences  from  the  image  of  the  nearly  block  diagonal 
matrix  Block  diagonal  matrices  were  not  unfamiliar  to  me.  for  they  had  played  an 
important  role  m  the  theoretical  work  I  had  done  on  causal  ordering  m  1952  and 
1953  The  mathematics  required  for  the  proofs,  which  was  fairly  standard,  would 
have  been  evoked.  i  think,  m  the  mind  of  any  mathematician  who  had  put  the 
problem  m  the  form  we  did  Our  results  were  rediscovered  by  some  Russian 
mathematicians  m  the  late  l960  s.  but  apparently  had  not  been  anticipated  earlier 

(Korolyuk.  Polischuk  and  Tomusyak.  A  A.,1969) 

Our  theorems  and  methods  (which  may  be  used  to  invert  matrices  that  are 
nearly  block  diagonal)  have  attracted  the  attention  of  numerical  analysts,  and  of 
natural  scientists  who  are  concerned  with  hierarchically  organized  systems.  The 

aggregation  method  we  introduced  has  also  now  been  recognized  to  be  closely 
related  to  the  so-called  "renormalization"  procedures  that  play  an  important  role  m 
several  parts  of  physics,  and  which  were  also  invented  quite  independently  of  ours 

Even  with  this  sketchy  account,  the  discovery  process  appears  quite 

unremarkable  The  problem  was  found  in  the  literature  (Goodwin  s  paper),  and  it  can 
be  represented  in  a  quite  standard  way  by  matrices  having  a  certain  special 

structure  The  metaphor,  by  showing  how  such  a  system  would  behave,  made  clear 
the  nature  of  the  theorems  to  be  proved  Although  nothing  is  revealed  ao^jt  the 
source  of  the  metaphor,  it  is  not  at  all  esoteric.  The  proofs,  while  intricate,  would 
not  pose  any  great  difficulty  for  a  professional  mathematician  A  case  of  normal 
problem  solving,  we  would  have  to  conclude 
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Finding  an  Explanatory  Model 

The  Isst  two  sections  provided  two  examples  ot  the  process  ot  finding  an 
explanatory  model  --  a  model  for  the  rank-freauency  relation  and  a  model  of  nearly 
decomposable  dynamic  systems  The  reader  may  be  interested  however  m  an 
example  a  little  closer  to  home  How  could  one  discover  an  explanatory  model  of 
human  problem  solving'’  One  answer  might  be  "By  observing  some  prot:>lem  solving 
behavior  closely  and  nducmg  the  model  directly  from  your  observations  " 

There  is  a  good  deal  of  merit  m  that  answer,  and  in  a  later  section  l  will  show 
that  something  like  that  happened  when  the  General  Problem  Solver  was  invented. 
But  even  in  this  case,  the  empirical  observations  were  not  the  sole  source  of 
information  that  g-.ded  the  discovery  The  inventors  also  had  some  notions  of  the 
shape  of  the  thing  they  were  looking  for 

Explanatory  theories  take  a  variety  of  forms.  For  example,  the  behavior  of 
gases  is  commonly  explained  by  supposing  that  they  consist  of  a  cloud  of  energetic 
particles,  interacting  with  each  other  in  accordance  with  the  laws  of  mechanics 
Magnetic  attraction  between  two  bodies  is  explained  by  a  field  of  magnetic  force  in 
the  space  between  them 

One  very  common  form  of  explanation,  in  both  natural  and  social  science 
employs  systems  of  differential  equations  or  difference  equations  to  determine  the 
values  of  the  time  derivatives  of  system  variables  At  any  given  time,  the  system  is 
supposed  to  be  in  a  specified  "state  "  and  the  differential  equations  then  determine 
to  what  state  it  will  be  moved  a  "moment"  later  Thus  m  mechanics,  the  state  is 
defined  in  terms  of  positions  and  velocities  and  the  differential  equations  show  how 
the  action  of  forces  to  produce  accelerations  brings  about  a  continuing  change  m 
state  through  time 

Building  an  explanatory  model  involves  a  choice  among  these  or  other 
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representations  of  the  phenomena  Will  it  be  a  particle  model  or  a  continuum 
model?  Will  it  represent  static  equilibrium  a  steady  state  or  dynamic  change’’  The 
representation  has  to  be  chosen  prior  to.  or  simultaneously  with  the  induction  of  the 
model  from  the  data 

When  Allen  Newell  Cliff  Shaw  and  i  began  the  construction  of  a  theory  to 
explain  problem  solving  around  1955  we  were  already  committed  to  a  representation 
In  fact,  it  was  our  recognition  that  such  a  representation  had  become  available  with 
the  invention  of  the  digital  computer  that  motivated  us  to  undenake  the  study  of 
human  thinking  A  reader  can  find  detailed  accounts  of  the  background  for  this 

recognition  in  Newell  and  Simon  1972  pages  873-886.  and  in  McCorduck.  especially 

Chapters  3  and  6. 

What  we  observed  -  we  have  told  the  story  before  (Newell  and  Simon,  1972)  -- 
was  that  the  program  of  a  computer  is  formally  equivalent  to  a  set  of  difference 
equations.  At  each  operation  cycle,  the  program  determines  the  new  state  of  the 

machine  as  a  function  of  its  previous  state  (the  contents  of  all  its  memories)  together 
with  any  new  input  it  has  received  Moreover  these  difference  equations  were  not 
limited  to  manipulating  numbers,  but  could  process  symbols  of  any  kind 

The  explanatory  task,  then,  was  to  fmd  a  dynamic  theory  of  the  processes  of 
problem  solving  in  the  form  of  a  computer  program  The  data  we  could  muster  on 
the  behavior  of  human  problem  solvers  had  to  be  examined  for  clues  as  to  the 
nature  of  that  orogram  This  requirement  piovided  very  strong  guidelines  both  for 
the  kinds  of  data  that  would  be  valuable  (preferably  data  that  followed  the  course  of 
problem  solution  as  closely  and  minutely  as  possible),  and  for  the  best  ways  d 

examining  the  data  (searching  out  the  succession  of  "actions'  the  problem  solve' 
executed,  and  the  cues  that  motivated  each  action) 

Of  course,  there  was  more  to  the  representation  than  simply  the  specification 
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that  It  be  a  comouter  program  u  had  to  contain  symbol  structures  that  couia 
represent  the  structures  m  human  memory  which  were  known  to  be  m  some  sense 
associative  The  important  point  was  that  there  was  a  continuing  two-way  interaction 
between  me  gradual  co^struciicn  o<  the  representation  and  the  construction  ot  the 
theory  tnat  useo  it  Sometimes  programming  convenience  lOr  necessity)  dictated 
cnoices  sometimes  psycnoiogicai  reauirements  Some  aspects  of  the  representation 
that  were  initially  conceived  mamiy  to  meet  programming  needs  (for  example,  the  iist- 
processing  languages  and  data  structures  m  the  form  of  lists  and  description  lists), 
were  later  seen  to  have  psychological  interpretations  as  networks  of  associations. 

The  empirical  part  of  me  undertaking  which  i  will  discuss  later  m  connection 
with  me  topic  of  experimental  design  went  nano  m  hand  with  me  design  of  the 
representation  of  me  explanatory  mode) 

Once  some  experience  had  been  gamed  with  information  processing  models  m 
me  form  of  computer  programs,  they  became  a  readi'y  available  tool  for  buldmg 
theories  of  other  asbects  c‘  human  thinking  So  Kotovsky  and  I,  interested  m 
explaining  simple  law-discovery  processes  as  a  first  step  toward  a  theory  of  scientific 
discovery  "naturally’  framed  our  model  as  a  computer  program  m  a  list  processing 
language,  capaoie  of  discovering  and  extrapolating  me  patterns  m  Thurstone  letter- 
sequence  problems  (Kotovsk,  and  Simon  m  S'mon  1979  ch  5  1.  5  2)  No 
alternative  reoresentations  were  even  cchside'’ed 

In  the  past  few  ,ears  with  jne  availability  ct  a  wnoie  new  menu  of  variants 
production  system.s  mode's  of  merncry  witn  spreadinQ  activation  connexionisi  m.coe's 
SCAR  me  PRCuCG  language  --  choices  of  representation  have  agam  beccm.e  an 
'mponani  and  difhcuJ  ca't  of  the  mod-.i-buiiding  process 


21 


The  Scientist  as  Probletn  Solver 


1  July  1 987 


Designing  Good  Excenments 

Experiments  are  supposed  to  be  aimed  at  testing  hypotheses  or  better  yet 
cnoosing  between  contending  hypotheses  f'cnticar'  experiments)  That  an  experiment 
meets  one  or  potn  ot  these  aims  is  neither  a  necessary  nor  a  sutt'Cient  condition  for 
Its  Demg  a  good  experiment 

It  IS  not  a  sufficient  condition  because  testing  weak-tea  hypotheses  of  the  form 
"variable  X  affects  variable  Y,"  or  its  negation,  is  not  usually  very  interesting,  and 
does  not  often  contribute  much  to  our  understanding  of  the  world  (But  if  I  continue 
in  this  vein  I  will  trespass  on  Allen  Newells  noted  "Twenty  Questions"  essay  (Newell 
1973) ) 

Testing  stronger  quantitative  hypotheses  (e  g..  the  periods  of  the  planets  are  as 
the  3/2  power  of  their  distances  from  the  Sun)  is  much  more  interesting,  and  very 
interesting  indeed  if  the  hypotheses  are  closely  connected  with  broad  explanatory 
theories  (e  g  .  with  the  inverse  square  law  of  gravitation) 

We  are  on  safer  ground  if  we  aim  experiments  at  testing  models  instead  of 
testing  hypotheses,  but  when  we  do  that,  we  must  remember  to  throw  away  the 
whole  standard  apparatus  of  significance  tests  which  is  no  longer  applicable.  (See 
Gregg  and  Simon,  and  the  references  cited  there  tm  Simon  1979.  ch  5.4).)  We 
must  also  remember  that  models  are  multi-component  creatures,  and  when  our  data 
don't  fit  a  model,  we  are  faced  with  a  difficult  diagnostic  task  to  determine  what  to 
change  --  or  whether  to  discard  the  entire  model 

So  much  for  sufficiency,  what  about  necessily7  is  model-testing  the  only 
reason  for  experimenting^  Surely  not  One  good  reason  (or  running  an  experiment 
--  or  fo''  spending  one  s  lime  |usi  observing  phenomena  closely  -  is  that  you  may  be 
surprised  The  best  things  that  come  out  of  experiments  are  things  that  we  didn  t 
expect  to  come  out  -■  especially  (hose  (hat  we  ccuidni  even  have  imagined  m 
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advance  as  ^.ssibiiities  Of  such  stuff  are  many  Mcbei  Prizes  made 

Lest  I  be  accused  of  advocating  planning  experiments  by  casting  dice  let  me 
suggest  mat  there  are  heuristics  for  planning  bom  kmds  of  experiments  experiments 
to  test  models  ana  experiments  to  generate  surprise  and  let  me  illustrate  me 
neunsucs  -vitn  some  examples  i  will  begm  with  the  more  traditional  model-testing 
category 

Testing  Models 

A  few  years  ago  i  found  occasion  to  begin  the  study  of  the  Chinese  language 
I  did  It  just  for  tun.  ana  because  I  planned  to  visit  China,  but  to  put  a  more  solemn 
face  on  things  I  called  it  "exposing  myself  to  new  phenomena  "  That  allowed  me 
to  do  some  of  i'  on  company  time  with  a  good  conscience  Finding  myself  in 
China  working  with  Chinese  psychologists,  we  decided  to  replicate  with  Chinese 
language  materials  some  standard  short-term  memory  experiments.  The  motive  was 

to  test  a  model  Does  Chinese  have  a  magical  numbern  And  is  it  seven'’  The 

answer  to  both  questions  was  'yes"  no  great  surprise 

Meanwhile.  i  had  learned  a  striking  fact  about  the  Chinese  language  (no 
Surprise  to  my  Chinese  colleagues,  but  a  surprise  to  me)  A  Chinese  college 

graduate  can  recognize  about  7  000  Chinese  characters  (hanzi)  Each  character  is 
pronounced  with  a  single  syllable  But  m  the  Chinese  language  there  are  only  about 
1.200  distinct  syllables  (even  taking  account  oi  tone  distinctions)  Hence,  on  average, 
there  are  about  six  homopnc'es  for  each  character 

Somehow  (intuition  or  recognition  at  workt  i  remembered  that  short-term, 

memory  was  generally  thought  to  be  acoustical  m  modality  but  only  because  cf 
Conrad  s  rather  indirect  evidence  that  errors  in  recall  generally  involved  simiiantv  m 
sound  rather  than  similarity  m  appearance  In  Chinese,  we  could  out  the  acoustical 
hypothesis  to  direct  test  After  establishing  that  the  STM  span  is  about  six  or  seven 
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unrelated  and  non-nomopnomc  characters  we  presented  the  same  subiects  with 
strings  ol  visually  distinct  homophomc  characters  The  result  was  dramatic  ••  the 
STM  spanned  dropped  to  about  two  confirming  Conrad  S  result  (Zhang  and  S'mon 
1985.  Vu.  Bolin  et  al  I985i 

Similar  methodological  predispositions  underlie  the  experiments  that  Bill  Chase 
and  I  did  on  memory  for  chess  positions  building  on  the  earlier  work  of  De  Groot 
and  others  (Chase  and  Simon  m  Simon  1979  ch  6  4  6  5)  Here  the  question  was 
whether  the  differences  m  chess  memory  between  experts  and  novices  could  be 
accounted  for  py  diifei'ences  m  their  vocabularies  of  "chunked"  chess  patterns  The 
answer  that  came  out  of  our  experiments  was  "qualitatively  yes.  but  quantitatively 
no."  an  answer  that,  if  slightly  disappointing,  was  much  sharper  than  if  we  had 
simply  asked  whether  experts  chunks  were  larger  than  novices' 

The  qualified  affirmative  answer  has  led  to  much  subsequent  research  which  is 
gradually  giving  us  a  more  precise  model  of  how  chunks  are  constructed  and 
organized  in  memory.  This  research  is  strong  represented  in  this  conference  by  the 
paper  of  Charness.  and  that  by  Ericsson  and  Staszewski,  while  closely  related 
questions  are  examined  by  Carpenter  and  Just  m  their  use  of  eye  movements  and  a 
memory  model  to  study  the  role  of  working  memory  m  reading 

The  experiments  with  Chase  on  chess  memory  like  those  with  Chinese 
characters,  were  designed  by  asking  what  quantitative  predictions  a  current  model 
made  and  how  we  could  make  the  measurements  necessary  for  testing  these 
predictions  The  problem  solving  search,  if  there  was  one  took  place  m  the  space 
of  the  characteristics  of  the  task  domain,  and  was  facilitated  by  looking  for 
Surprising"  or  "interesting"  features  of  the  domain  In  the  Chinese  language  case, 
the  surp'’'sing  feature  was  found  first  and  the  model  it  was  relevant  to  was  found 
second  In  the  chess  case  the  order  was  reversed 
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The  experiments  /usi  described  all  have  an  experimental  and  a  control 
condition,  just  as  a  well-designed  experiment  is  supposed  to  in  the  Chinese 
language  experiments  we  compared  homopnonic  with  npn-nomophonic  strings  of 
characters  in  the  chess  experiments,  we  compared  the  performance  of  experts  wun 
the  performance  of  novices,  and  chess  positions  from  well-played  games  with  random 
positions  The  expert-novice  dichotomy  nas  also  served  me  m  good  stead  m  some 
more  recent  experiments  on  problem  solving  m  physics  (Simon  and  Simon.  1978 
Larkin  et  ai  .  1980).  and  has  been  used  by  Hayes  (this  volume)  and  his  colleagues  m 
research  on  writing  An  incidental  benefit  of  using  this  paradigm  is  that  being  able 
to  point  to  clearcut  experimental  and  control  conditions  seems  to  soothe  the  savage 
breast  of  reteree  and  editor 

Problem  Isomorphs 

To  conclude  my  list  of  experimental  manipulations.  l  will  mention  just  one  other, 
which  has  provided  us  with  almost  unlimited  mileage  -  the  idea  of  problem 
isomorphs.  Its  history  is  as  obscure  as  that  of  many  of  the  other  things  I  have 

been  talking  about  i  think  I  invented  the  idea  of  problem  isomorphs  about  1969.  or 

a  little  earlier;  for  l  do  not  have  any  evidence  of  earlier  mention  by  myself  or  anyone 
else  I  have  a  conjecture  about  its  antecedents  but  it  is  a  reconstruction,  not  a 

recollection,  although  John  Michon,  without  prompting  corroborated  it 

Saul  Amarel  was  one  of  the  first  researchers  m  artificial  intelligence  to  point 
out  that  changing  the  representation  of  a  problem  -  the  problem  space  and 
operators  -  could  sometimes  greatly  facilitate  its  solution  Amabel  Newell,  and  i 

participated  m  a  semester-long  seminar  at  CMU  m  1966.  the  mam  topic  of  which  was 
problem  representation  Now  it  is  only  a  small  step  (at  least  by  hindsight)  from  me 
idea  that  a  subject  can  solve  a  problem  easily  by  finding  the  right  representation  to 
the  Idea  that  an  experimenter  can  make  a  problem  harder  or  easier  for  a  subject  by 
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presenting  it  m  one  or  another  guise 

So  much  for  the  antecedents  Soon  problem  isomorphs  ••  problems  with 
identical  task  domains  and  legal-move  operators,  but  described  by  different  sets  of 
words  "  were  a  topic  of  discussion  m  the  Understand  Seminar  (alias  the  Cognitive 
Science  Seminar)  which  has  run  weekly  m  the  Psychology  Department  for  twenty 
years  The  first  example  was  number  scrabble  an  isomorph  of  tic-'ac-toe  and  John 
Michon  then  added  another  member.  JAM.  to  this  set  John  R  Hayes  rapidly 
became  the  most  prolific  and  ingenious  designer  of  problem  isomorphs.  providing  us 
with  somewhere  between  a  dozen  and  two  dozen  isomorphs  of  the  Tower  of  Hanoi 
puzzle,  most  of  which  have  been  used  in  one  or  more  experiments  (Hayes  and 
Sir^on,  in  Simon.  1979  ch  7  1-7  3). 

We  have  used  isomorphs  to  discover  what  characteristics  of  a  problem,  other 
than  the  size  of  the  task  domain,  account  for  its  difficulty.  Early  work  in  problem 
solving,  our  own  included,  had  focused  on  the  combinatorial  explosion  of  search  as 
the  mam  source  of  problem  difficulty  Yet  we  had  found  that  the  Tower  of  Hanoi, 
with  a  relatively  small  and  easily  exhaustible  domain,  and  the  Missionaries  and 
Cannibals  puzzle,  with  a  tmy  one.  could  occupy  human  adults  for  fifteen  minutes  or 
a  half  hour  before  they  found  a  solution 

The  Idea  that  only  the  size  of  the  task  domain  could  affect  problem  difficulty 
sometimes  died  hard  One  referee  for  a  funding  agency  gave  a  project  proposal  low 
marks  because  he  or  she  thought  that,  on  these  grounds  our  experiments  could 
have  only  negative  results,  since  all  isomorphs  must  be  of  the  same  difficulty  (At 
the  time  we  were  told  of  this  objection,  we  had  already  demonstrated  differences  m 
average  solution  times  to  various  isomorphs  of  the  Tower  of  Hanoi  m  the  ratio  of  i6 
to  1  )  Our  theoretical  account  of  problem  difficulty  is  still  very  incomplete,  although 
we  have  been  able  to  model  some  of  the  phenomena  (Kotovsky,  Hayes  and  Simon 
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1985  Kotovsky  and  Fallside  this  volume)  (The  UNDERSTAND  program,  for  example, 
constructs  different  representations  of  problems  when  presented  with  different 
isomorphs  (Hayes  and  Simon,  m  Simon.  1979.  cn  7  1)) 

fne  experimental  straieg,  here  ’S  clear  deriving  from  the  single  idea  of 

isomorphism  ••  and  mat  idea  has  at  least  a  plausible  lineage  The  power  of  the 
idea  would  be  enhanced  if  we  had  more  systematic  ways  of  designing  isomorphs  with 
specific  features  designed  m  advance  to  test  particular  putative  sources  of  difficulty 

Experimenting  Without  an  Independent  Variable 

The  experiments  described  up  to  this  point  all  compare  performance  under  two 
or  more  different  conditions  --  they  all  involve  manipulation  of  an  independent 

variable  When  l  examine  my  publications  beyond  the  limited  set  already  mentioned, 
I  find  to  my  embarrassment  that  this  fundamental  condition  for  sound  experirricntation 
is  seldom  met  m  them  What  have  I  been  up  to?  What  can  I  possibly  have 

learned  from  ill-designed  experiments^  The  answer  (it  surprised  me)  is  that  you  can 

test  theoretical  models  without  contrasting  an  experimental  with  a  control  condition 
And  apart  from  testing  models,  you  can  often  make  surprising  observations  that  give 
you  ideas  for  new  or  improved  models 

Let  me  start  with  an  example  of  the  latter  kmd  Many  summers  ago  (about 

1965)  Jeffery  Paige  and  I  decided  to  take  thinkmg-aioud  protocols  from  high-schooi 

students  solving  algebra  word  problems  (Paige  and  Simon  m  Simon  1979.  ch  4  4) 
Our  mam  motivation  l  think  was  just  to  see  how  they  did  it  --  what  processes  they 
used  Perhaps  we  had  m  mind  comparing  their  behavior  with  Bobrow  s  STUDENT 
program  (Bobrow  1968),  which  solved  such  problems  Or  perhaps  we  thought  we 
might  build  a  program  ourselves  that  would  do  a  better  job  of  s'muiatmg  the  human 

processes  if  those  were  our  intentions,  my  memory  does  not  retain  them 

Jeff  conceived  of  a  fine  idea  (at  least,  i  have  always  remembered  it  as  his) 
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We  constructed  some  "impossible"  problems  -•  problems  that  could  not  be  given  a 
real  physical  interpretation  because  their  solutions  involved  boards  of  negative  length 
or  nickels  that  were  worth  more  than  dimes  We  then  asked  our  subjects  to  set  up 
me  eouations  corresponding  to  the  problem  statements,  but  not  to  solve  them 

The  outcome  was  wholly  unanticipated  Our  subjects  fell  into  three  groups, 
rather  consistently  over  the  set  of  three  problems.  Some  set  up  the  equations  that 
corresponded  literally  with  the  verbal  statements  of  the  problems.  Some  translated 
the  problems  inaccurately,  always  ending  up  with  equations  that  corresponded  to  a 
reailzeable  physical  situation  Some  said,  "Isn't  there  a  contradiction^"  -  meaning, 
"I  draw  inferences  from  the  problem  statements  that  conflict  with  my  knowledge  of 
the  real  world." 

Because  we  were  trying  to  get  as  dense  a  set  of  data  as  we  could,  in  order 
to  trace  processes  m  detail,  we  had  asked  the  subjects  both  to  think  aloud  and  to 
draw  diagrams  of  the  problem  situations  The  diagrams  drawn  by  subjects  in  the 
first  group  were  generally  incomplete  and  unmtegrated,  and  did  not  '•eveal  the 
"contradiction."  The  diagrams  drawn  by  subjects  in  the  second  group 
misrepresented  the  situations  m  just  the  way  their  equations  did  -•  so  as  to  make 
them  physically  reaiizeabie  The  subjects  m  the  third  group  drew  diagrams  that 
revealed  the  contradictions 

The  direction  of  the  causal  arrow  is  not  clear,  but  one  can  take  these  results 
as  at  least  presumptive  evidence  that  subjects  m  the  second  and  third  groups  used 
imagery  to  represent  the  information  from  the  word  problems  before  translating  into 
the  language  of  algebra  Subjects  m  the  first  group  gave  evidence  of  translating 
directly  to  equations  using  only  syntactic  information 

With  this  kind  of  information  in  hand,  one  can  begin  to  construct  models  for 
the  simulation  of  these  sorts  of  behavior,  and  to  explore  what  other  predictions  could 
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be  made  about  systems  behaving  m  these  ways  The  ISAAC  program,  written  by 
Gordon  Novak  to  solve  physics  problems  presented  in  natural  language  is  an 
example  of  a  System  that  uses  an  internal  diagram  of  the  problem  situation  to 
mediate  between  the  verbal  stimulus  and  the  eauations  it  finally  constructs  (Novak. 
1976)  The  UNDERSTAND  program  that  John  R  Hayes  and  i  constructed  around 
1972.  to  show  how  verbal  problem  instructions  could  be  converted  into  inputs 
appropriate  for  a  GPS-like  problem  solver,  also  borrowed  this  insight  from  the  algebra 
experiments  (Hayes  and  Simon,  m  Simon.  1979.  ch  7  1) 

But  the  most  massive  set  of  examples  of  the  experimental  strategy  of  "just 
looking'  is  to  be  found  in  Human  Problem  Solving."  Density  of  data  was  ’he  name 
of  the  game,  and  protocol  analysis  the  way  of  playing  it.  In  1956.  the  Logic  Theorist 
(LT)  had  demonstrated  the  feasibility  of  solving  difficult  problems  by  highly  selective 
heuristic  search  (Newell  and  Simon.  i956).  But  is  that  the  way  people  did  it?  The 
General  Problem  Solver.  LT  s  successor,  was  our  answer  -  a  heuristic  search  system 
that  used  means-ends  analysis  as  its  principal  heuristic  (Newell  and  Simon.  1972). 

Both  Al  Newell  and  I  agree  that  the  core  of  GPS  was  extracted  directly  from  a 
particular  protocol  that  we  can  identify  We  also  agree  as  to  the  week  m  the 
Summer  of  1957  when  it  was  done  On  the  details,  the  evidence  is  not  wholly 
concordant,  but  sometime,  when  we  have  leisure  to  examine  the  papers  we  have 
preserved,  we  may  get  it  all  straightened  out  (See  McCorduck.  page  2i2)  The  mam 
lesson  IS  clear  GPS.  a  theory  of  human  problem  solving,  was  extracted  by  direct 
induction  from  the  thinking  aloud  protocol  of  a  laboratory  subieci  without  benefit  of 
5n  "excerimenial"  and  a  "control"  condition 

What,  in  addition  to  luck,  entered  into  the  result'^  First,  as  i  have  pointed  out 
in  an  earlier  section  we  already  knew  that  we  wished  to  represent  our  model  as  a 
computer  program  in  a  iist-processing  language  Second,  a  data  gathering  method 
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was  used  that  obtained  the  densest  record  of  the  subiect  s  behavior  that  we  knew 
how  to  get  We  were  able  to  discover  what  he  had  done  each  few  seconds  of  time 
during  which  he  worked  on  the  task  Third,  some  care  had  been  taken  m  selecting 
the  task  '!  ^cc  aireaay  been  used  by  O  K  Moore  and  his  colleagues  at  Yale  ano 
we  had  access  to  both  their  experience  and  their  data  The  task  was  symbolic 
hence  made  for  easy  verbalization  and  seemed  to  call  for  a  minimum  of  pictorial 
visualization  it  was  a  harp  enough  task  to  evoke  genuine  problem  solving  behavior 
from  intelligent  subjects 

Application  of  these  criteria  to  the  selection  of  problem  solving  tasks  accounts 
for  a  substantial  fraction  of  the  knowledge  that  has  been  collected  about  problem 
solving  processes  during  the  past  thirty  years,  and  a  substantial  part  of  the 
theoretical  efforts  that  have  succeeded  in  building  models  to  account  for  behavior  m 
many  kinds  of  tasks  The  metaphor  of  chess,  cryptarithmetic  and  the  Tower  of 
Hanoi  serving  as  the  green  peas.  Drosophila  and  E,  coii  of  cognitive  science  is  as 
near  to  literal  truth  as  it  is  to  fancy 

Do  these  experiments  really  lack  independent  variables'!*  Can  t  we  consider  the 
task  domain  or  the  subject  to  be  just  that"^  Of  course  we  can.  but  to  no  particular 
end  The  principal  knowledge  we  gained  from  these  experiments  did  not  come  out 
of  comparisons  between  tasks  or  subjects  it  came  out  of  painstakingly  analyzing 
individual  protocols  and  inducing  from  them  the  processes  that  problem  solvers 

employed  m  their  work  Once  this  had  been  done,  we  could  test  the  generality  of 

our  results  by  comparing  over  tasks  and  over  subjects  But  detailed  longitudinal 
analysis  of  the  behavior  of  single  subjects  was  the  foundation  stone  for  the 
information  processing  theories  we  have  built  of  what  goes  on  m  human  problem 

solving 

If  the  methodology  troubles  us.  it  may  be  comforting  to  recall  that  detailed 
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longitudinal  analysis  of  me  behavior  o<  a  smgie  solar  system  was  me  foundation 
stone  for  Kepler  s  Laws,  and  ultimately  (or  Newton  s  Perhaos  it  is  not  our 
methodology  that  needs  revising  so  much  as  the  standard  textbooK  methodology 
which  perversely  warns  us  agamst  running  an  experiment  until  precise  hypotheses 
have  been  formulated  and  experimental  and  control  conditions  defined  How  do  sucn 
experiments  ever  create  surprise  -  not  just  the  all-too-common  surprise  of  having  our 
hypotheses  refuted  by  facts,  but  the  deiight-provokmg  surprise  of  encountering  a 
wholly  unexpected  phenomenon^  Perhaps  we  need  to  add  to  the  textbooks  a 
chapter,  or  several  chapters,  describing  how  basic  scientific  discoveries  can  be  made 
by  observing  the  world  intently,  in  the  laboratory  or  outside  it,  with  controls  or  without 
them,  heavy  with  hypotheses  or  innocent  of  mem 

The  Scientist  as  a  Satisficer 

My  economist  friends  have  long  since  given  up  on  me.  consigning  me  to 
psychology  or  some  other  distant  wasteland  if  I  cannot  accept  the  true  faith  of 
expected  utility  maximization,  it  is  not  the  fault  of  my  excellent  education  in 
economics  ••  m  fact,  the  education  was  repeated  tour  times  often  enough  even  for  a 
slow  learner  First  as  a  high  school  student.  I  read  me  works  of  Richard  Ely  and 
Henry  George  m  order  to  meet  the  arguments  of  opposing  debating  teams  on  such 
issues  as  the  tariff  or  the  single  tax  Then,  at  the  University  of  Chicago.  i  learned 
price  theory  from  Henry  Simonds  and  Walrasian  eauiiibnum  and  econometrics  from 
Henry  Schultz 

Next,  at  Berkeley  my  colleagues.  Kenneth  May  and  Ronald  Shepard,  students 
of  Griffitn  Evans,  revealed  to  me  the  inference-drawing  powers  of  the  second-order 
conditions  of  maximization,  while  I  learned  about  Neyman-Pearson  statistics  from  Jerzy 
Neyman  himself  Finally,  on  returning  to  Chicago.  i  was  exposed  to  Samueison  s 
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Pounaauons  and  Hicks  on  value  m  me  brilliant  discussions  at  me  Cowles  Commission 
seminars  among  Jascha  Marschak.  Tjalling  Koopmans  Ken  Arrow.  Larry  Klein  Franco 
Modigliani.  Gerard  Debreu  and  other  superbly  keen  and  well-informed  mmds 

Alas  It  did  not  take  My  traumatic  exposure  m  1935  to  me  budgeting  process 
in  me  Milwaukee  recreation  department  had  immunized  me  against  the  idea  mat 
human  bemgs  maximize  expected  utility  and  had  made  of  me  an  incorrigible 

satisficer  And  that  same  imprinting  experience  supplied  me  with  the  problem  -  me 
cornucopia  of  problems  ••  that  has  kept  me  occupied  ever  since.  I  have  sketched 
here  the  theory  of  scientific  discovery  to  which  my  study  of  these  problems  has  led 

me  It  is  not  a  theory  of  global  rationality,  but  a  theory  of  human  limited 

computation  m  the  face  of  complexity  It  views  discovery  as  problem  solving,  and 
problem  solving  as  heuristic  search  and  heuristic  search  as  the  only  fit  activity  for  a 
creature  of  bounded  rationality 

Some  scientists  believe  that  theories  should  be  judged  by  their  ability  to  make 

correct  predictions  This  paper  provides  some  tests  of  the  predictive  power  of  this 

problem-solving  theory  of  discovery  The  anecdotes  I  have  provided  from  my  own 

scientific  life  are  instances  where  it  gives  a  pretty  good  account  of  the  processes 

that  are  visible  in  my  research 

It  describes  me.  like  KEKADA.  formulating  a  new  problem  m  response  to  my 
surprise  at  encountering  an  unexpected  phenomenon  it  traces  my  BACON-like 
progress  toward  discerning  a  (awful  regularity  m  data  and  the  evocation  of 
knowledge,  m  expert-system  style,  to  fmd  an  explanation  for  me  regularity  it 

accounts  for  my  use  of  diagrams  to  gam  a  grasp  of  complex  phenomena  m  a 

dynamic  system  It  illuminates  how  the  availability  of  representations  and  the  invention 
of  new  ones  has  influenced  my  efforts  to  construct  explanations  it  characterizes  a 
number  of  my  strategies  for  designing  experiments,  and  perhaps  even  explains  why  i 


32 


The  Scientist  as  Problem  SoKer 


1  July  1987 


am  frequently  unconcerned  about  such  things  as  "experimental  controls"  or  even 
independent  variables 

Of  course  I  am  exercising  poet'C  license  m  talking  of  predictions  A 

comprehensive  SIMPLE  SIMON  has  not  been  programmed,  only  pieces  of  him  exist 
It  would  be  more  defensible  to  talk  of  explanatory  accounts  rather  than  predictions 
But  you  will  not  be  misled  by  the  metaphor,  which  is  as  useful  as  one  can  expect  a 
metaphor  to  be 

The  information  processing  theory  of  discovery  that  1  have  been  describing  has 
one  other  virtue.  It  is  not  only  a  descriptive  theory,  but  a  normative  one  as  well. 

Not  only  does  it  predict  (explain)  my  behavior  successfully,  but.  unbeknownst  to  me. 

it  has  served  me  for  fifty  two  years  as  a  reliable  set  of  heuristics  for  conducting 
research  Quite  unwittingly,  l  have  been  following  the  instructions  of  BACON,  of 

STAHL,  of  GLAUBER,  of  DALTON,  and  of  KEKADA.  I  couldn't  have  had  better 
guidance 

However,  one  heuristic  that  has  been  of  first  importance  to  my  work  is  missing 
from  these  programs.  I  will  mention  it.  because  you  too  may  find  it  useful.  If  you 
want  to  mar.e  interesting  scientific  discoveries,  be  sure  to  acquire  as  many  good 
friends  as  possible,  who  are  as  energetic,  intelligent,  and  knowledgeable  as  they  can 
be.  Form  partnerships  with  them  whenever  you  can  Then  sit  back  and  relax.  You 
will  find  that  all  the  programs  you  need  are  stored  m  your  friends,  and  will  execute 
productively  and  creatively  as  long  as  you  don  t  interfere  too  much 
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